Images and video come in a large variety of formats. For many applications, a conversion between formats is performed. A high-quality, low-cost method for converting the video signals is very useful for such applications as (i) converting interlaced NTSC video at 30 fields/second to progressive video with a similar or larger horizontal and vertical resolution at 60 frames/second for display on progressive televisions, (ii) performing a high-quality “zoom” function on either interlaced or progressive video and (iii) increasing the horizontal and/or vertical resolution of progressive or interlaced video and images.
Existing solutions for video deinterlacing include (i) bob (i.e., vertical spatial filter), (ii) weave (i.e., temporal filter), (iii) VT-filtering (i.e., vertical spatial filter combined with temporal filter), (iv) motion-adaptive and motion-compensated techniques and (v) edge-based spatial filtering. The existing video techniques which are not temporal in nature can also be applied to image upconversions (i.e., vertical and edge-based spatial filtering). Horizontal and edge-based spatial filtering are used for horizontal upsampling of images and video.
Vertical filtering is known to produce temporal flickering artifacts in video and significantly reduced vertical detail in both images and video. Odd and even parity lines are alternately blurred in the video and interpolated in a vertical direction only from adjacent lines. The lack of vertical detail is particularly noticeable for sharp edges.
Temporal filtering is known to produce “jaggies”. Jaggies a re interlace artifacts that are extremely objectionable for moving objects.
The VT-filtering is a fixed (i.e., non-adaptive) filtering that combines a high pass version of a previous opposite parity field with a lowpass interpolation of a missing line from a current field. The VT-filtering is a low-cost line-based process that is cost effective to implement in silicon, but is known to produce temporal artifacts. The artifacts can include trailing edges or “edge ghosts” from previous fields that appear behind moving objects.
Motion adaptive techniques make pixel-level, block-level, and/or picture-level decisions about whether to use weave or bob or a blended combination of weave and bob for particular pixels, blocks and/or pictures. Weave is the best option for still portions of video, while a poor choice for moving areas. Hard block-level decisions can lead to objectionable blocking artifacts. However, more advanced motion adaptive deinterlacing techniques that combine weave and bob suffer mainly from relatively poor performance for moving video, for which all of the drawbacks of bob are encountered. For stationary regions, however, the flickering artifact suffered by bob can be greatly reduced.
Motion compensated techniques operate in a similar manner to motion adaptive techniques. A consideration for motion compensated techniques is that motion compensated pixels are chosen from a previous opposite parity field rather than always using the co-located pixels from the previous opposite parity field to replace the missing pixels in a progressive frame that is formed from the current field (i.e., weave). An advantage of the motion compensated technique is that moving video that can be well estimated and motion-compensation will yield far superior deinterlaced video. A disadvantage of motion compensated technique is that motion estimation is typically far more expensive than any of the previously mentioned techniques. Furthermore, if motion estimation fails on the video sequence (i.e., highly irregular motion, non-smooth motion fields or various lighting effects may cause motion estimation to fail), then motion compensated techniques can be no better than less complex methods. Furthermore, even when motion estimation is successful, the amount of high-frequency information that can be transferred from the previous opposite parity field to the estimate of the missing lines for reconstruction a progressive frame from the current field depends upon the sub-pel motion between the two fields. In the worst case, objects can move by an integer number of pels plus exactly one-half pel in the vertical direction in the temporal interval that passes between the previous and current field. Therefore, no additional high-frequency vertical information for the missing lines of the current field can be gleaned from the previous field through the motion compensated estimate from the previous field. In practice, however, motion compensated deinterlacing greatly increases vertical detail, while greatly reducing flickering artifacts on a broad range of video, such that the main drawback is complexity.
Edge-based spatial filtering operates on only the current field and is capable of producing a far superior estimate of the pixels from the missing lines than what is possible with vertical filtering only. To a lesser extent than vertical filtering, edge-based spatial filtering also suffers from lack of vertical detail. In particular, high frequency textures that lack edges will not be improved over simple vertical filtering.